<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<html><head><title>Programming in Lua : 21.2.1</title>
<link rel="stylesheet" type="text/css" href="pil.css">
</head>
<body>
<p class="warning">
<A HREF="contents.html"><IMG TITLE="Programming in Lua (first edition)" SRC="capa.jpg" ALT="" ALIGN="left"></A>This first edition was written for Lua 5.0. While still largely relevant for later versions, there are some differences.<BR>The third edition targets Lua 5.2 and is available at <A HREF="http://www.amazon.com/exec/obidos/ASIN/859037985X/theprogrammil3-20">Amazon</A> and other bookstores.<BR>By buying the book, you also help to <A HREF="../donations.html">support the Lua project</A>.
</p>
<table width="100%" class="nav">
<tr>
<td width="10%" align="left"><a href="21.2.html"><img border="0" src="left.png" alt="Previous"></a></td>
<td width="80%" align="center"><a href="contents.html"><font face="Helvetica,Arial,sanserif">
<font color="gray">Programming in </font><font color="blue"> Lua</font>
</font></a></td>
<td width="10%" align="right"><a href="21.2.2.html"><img border="0" src="right.png" alt="Next"></a></td>
</tr>
<tr>
<td width="10%" align="left"></td>
<td width="80%" align="center"><a href="contents.html#P3">Part III. The Standard Libraries</a>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
<a href="contents.html#21">Chapter 21. The I/O Library</a></td>
<td width="10%" align="right"></td></tr>
</table>
<hr/>
<p><h3>21.2.1 &ndash; A Small Performance Trick</h3>

<p>Usually, in Lua, it is much faster to read a file as a whole than
to read it line by line.
However, sometimes we must face some big files
(say, tens or hundreds megabytes)
for which it is not reasonable to read them all at once.
If you want to handle such big files with maximum performance,

the fastest way is to read them in reasonably large chunks
(e.g., 8 KB each).
To avoid the problem of breaking lines in the middle,
you simply ask to read a chunk plus a line:
<pre>
    local lines, rest = f:read(BUFSIZE, "*line")
</pre>
The variable <code>rest</code> will get the rest of any line
broken by the chunk.
We then concatenate the chunk and this rest of line.
That way, the resulting chunk will always break at line boundaries.

<p>A typical example of that technique is this implementation of <code>wc</code>,
a program to count the number of characters, words, and lines in
a file:
<pre>
    local BUFSIZE = 2^13     -- 8K
    local f = io.input(arg[1])   -- open input file
    local cc, lc, wc = 0, 0, 0   -- char, line, and word counts
    while true do
      local lines, rest = f:read(BUFSIZE, "*line")
      if not lines then break end
      if rest then lines = lines .. rest .. '\n' end
      cc = cc + string.len(lines)
      -- count words in the chunk
      local _,t = string.gsub(lines, "%S+", "")
      wc = wc + t
      -- count newlines in the chunk
      _,t = string.gsub(lines, "\n", "\n")
      lc = lc + t
    end
    print(lc, wc, cc)
</pre>

<hr/>
<table width="100%" class="nav">
<tr valign="top">
<td width="90%" align="left">
<small>
  Copyright &copy; 2003&ndash;2004 Roberto Ierusalimschy.  All rights reserved.
</small>
</td>
<td width="10%" align="right"><a href="21.2.2.html"><img border="0" src="right.png" alt="Next"></a></td>
</tr>
</table>


</body></html>

